Enhancing Cloud Services through Multitier Workload Analysis

Services computing is penetrating IT and computing technology at every level, encompassing the Web, the cloud, big data, business process modeling, and more. One feature that distinguishes cloud computing from conventional distributed computing is its hierarchical organization of computing capabilities as services, represented by infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). In “Variations in Performance and Scalability: An Experimental Study in IaaS Clouds Using Multitier Workloads” (IEEE Trans. Services Computing, vol. 7, no. 2, 2014, pp. 293–306), Deepal Jayasinghe and his colleagues describe an approach to enhancing IaaS cloud services through multitier workload analysis. IaaS promises considerable economic benefit because applications can be encapsulated in their own virtual machines and “run anywhere.” This application portability across many clouds theoretically enables users to choose the most cost-effective IaaS service provider. Several production cloud environments can achieve portability very quickly, with typical whole system setup times on the order of hours instead of the days required in nonvirtualized environments. Although typical applications can be brought up quickly in computing clouds, the complexity of modern n-tier applications can’t be completely masked by a single virtualization layer. Several macro-level indicators reveal serious challenges in making largescale, mission-critical applications run equally well in different clouds. First, average datacenter utilization has been reported at very low levels over the years, with a Gartner survey reporting 18 percent average utilization; Google reports about 30 percent for a mixed workload combining longrunning batch jobs with Web-facing applications. Second, unexpectedly long response-time requests (several seconds), at a relatively low average of 50 to 60 percent CPU utilization, have been associated with very short bottlenecks that last only a fraction of a second. Based on extensive experimental analysis, the authors report on the differences they’ve found among six IaaS virtualized cloud environments by running standard benchmark applications— such as RUBBoS and CloudStone—with similar or the same configuration settings. They compared performance and scalability variations in three representative public cloud infrastructures: EC2, Open Cirrus, and Emulab. An interesting discovery from large-scale experiments is that for the RUBBoS n-tier application benchmark, the bestperforming configuration in Emulab can become the worstperforming configuration in EC2 due to a combination of hardware and software component differences, even though the RUBBoS implementation has been ported with minimized changes. The authors also compared the nontrivial differences among three mainstream hypervisors— Xen, VMware, and KVM— in a controlled environment. Their discoveries show significant differences among six modern IaaS cloud infrastructures and providers. Specifically, functional portability—which is routinely demonstrated—doesn’t necessarily imply performance portability; the latter requires careful study, measurement, and analysis.